Local estimates for entropy densities in coupled map lattices 
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We present a method to derive an upper bound for the entropy density of coupled map lattices 
with local interactions from local observations. To do this, we use an embedding technique being a 
combination of time delay and spatial embedding. This embedding allows us to identify the local 
character of the equations of motion. Based on this method we present an approximate estimate of 
the entropy density by the correlation integral. 



Many natural phenomena involving a huge number of 
degrees of freedom emerge from spatially extended sys- 
tems (SESs), with translationally invariant equations of 
motion and spatial coupling. Thus an understanding of 
these kinds of systems is vital for the understanding of 
nature. Only for very few of them there exist models 
from first principles which allow us to explore them by 
analytical or numerical means. Many systems can not 
be satisfactorily modeled, so that one has to rely on the 
study of experimental observations. This usually leads 
to time series analysis, which has gained new stimulus 
in recent years through the concept of phase space re- 
construction. This approach is quite successful for low 
dimensional chaotic systems |^,^. There one can ob- 
tain characteristic quantities like Lyapunov exponents, 
fractal dimensions and entropies directly from the mea- 
surements. In some cases it is even possible to construct 
reasonable model equations from the data. 

Natural SESs are typically continuous in time and 
space and thus described by partial differential equations 
(PDEs). Nevertheless, a PDE can be approximated by 
a coupled map lattice (CML), i.e. a model discrete in 
time and space. CMLs were used as paradigmatic mod- 
els to study general properties of spatio-temporal chaotic 
systems. A fundamental question concerning SESs is 
whether it is possible to extract characteristic quanti- 
ties from local measurements. This problem is still un- 
solved in the realm of time series analysis. There were 
a considerable number of attempts, especially for the di- 
mension density Nevertheless, the outcome was 
far from being satisfactory. Common to all these works 
is to use either time delay or purely spatial embedding 
for reconstructing the dynamical states or the invariant 
measure. The general problem is that local measure- 
ments can only reconstruct finite dimensional subspaces, 
though the whole phase space is infinite dimensional, in 
principle. Therefore, these subsystems are never fully 
deterministic, since they are coupled to the unobserved 
part of the phase space and are thus open systems. 



However, in systems with local, e.g. nearest neigh- 
bour, interactions, the equations of motion include only 
a few local variables. Regarding only these variables 
the "local" future is governed by deterministic laws, 
while the whole dynamics in the measured subspace re- 
mains "stochastically" driven. The main idea we want 
to present is that the local deterministic structure can 
be exploited to estimate a dynamical entropy which will 
turn out to be a good approximation for the entropy den- 
sity of the whole (mainly unobserved) system. 

The general CML we want to study here is given by 



Xi{n + 1) = (1 - (Tr - ai)f{xi{n)) 

+cr;g(xi_i(n)) -f CTrg(xi+i(n)) , 



(1) 



where i represents the position in space and n the time. 
CTr and ai are spatial coupling constants. Concerning the 
coupling, typically two kinds of systems are studied: (1) 
(Jr = cTi = a which we will refer to as the (symmetric) 
diffusive coupling and (2) (Tr(i) = , f7;(r) = cr the uni- 
directional coupling. The map / is a chaotic map which 
describes the local dynamics. We will use the tent map 
f{x) = 1 — 2\x — 1/2| throughout this paper, g is an in 
principle arbitrary function. We use either g ~ f which 
gives rise to a nonlinear coupling or g{x) — x which re- 
sults in a linear coupling. 

It is commonly observed that the dynamics of these 
CMLs is extensively chaotic in suitable parameter re- 
gions. Extensive means that the attractor dimension and 
the KS-entropy are proportional to the system size, here 
the number N of lattice sites. Thus it is possible to de- 
fine dimension and entropy densities as intensive quanti- 
ties. Usually these densities are estimated via the Pesin 
identity or the Kaplan- York formula, respectively, by cal- 
culating the Lyapunov spectrum using the model equa- 
tions. In some cases this is even possible if the equations 
of motion are unknown ]To[ |. 

In previous attempts to calculate entropies and 
dimensions from observed time series either pure 
spatial embedding with state vectors of the form 



1 



{xi{n), . . . , Xi+i-i{n)) or temporal embedding with the 
vectors {xi{n), . . . , Xi{n+m—l)) were used. Since the dy- 
namics for all these embeddings is non-autonomous with 
respect to equation the corresponding projections 
of the invariant measure of the CML contain stochastic 
components so that the entropy diverges In this 

letter we introduce a novel embedding procedure ( "pyra- 
mid embedding" ) which does not increase the dimension 
of the reconstructed measure when the dimension of the 
involved subspace is increased. This will enable us to 
compute the finite entropy related to the deterministic 
part of the dynamics in the subspace. 

Consider a partitioning of a subspace spanned by 
the components of a state vector s with a rectangular grid 
of mesh size e. Via the invariant measure we can assign 
a probability pi to every cell and define the entropy 



H{s,e) ^ -y^P» lnp» 



(2) 



For sufficient small e its e-dependence is given by 

H{s,e) cc-Dlne , (3) 

where D is the information dimension of the projection 
of the invariant measure into the subspace spanned by 
s. Let us choose a second subspace Fj- spanned by the 
components of another state vector t satisfying 



(4) 



which implies that the constraint F is determined by 
equation (|l|). H{t,s,e) denotes the entropy of the joint 
probability pij for the system being in cell i of Fj- and 
in cell j of F^-. The projection of the invariant measure 
in the enlarged state space has the identical information 
dimension D due to the constraints (^. Therefore the 
conditional entropy 



h{t\s, e) = H{t, s, e) - H{s, e) 



(5) 



will become independent of e for sufficient small e. 

In the following we will use a abbreviated symbolic 
representation for the states appearing as arguments in 
the entropies. The pure spatial state of / neighbouring 
sites will be denoted by 



{xi{n),X2{n), . . . ,xi_i{n),xi{n)) 



This notation will allow us to write states which combine 
spatial and temporal embedding in a compact way. Note 
that we can omit the time and space indices because of 
the stationarity and translation invariance. 

The simplest way to choose state vectors s and t ful- 
filling the constraint is s = {xi{n) , Xi+i{n)) and 
t = {xi(n + 1)) for unidirectional coupling and s = 



{xi-i(n),Xi{n),Xi+i{n)) and t = {xi{n + 1)) for diffu- 
sive coupling, respectively. Using the symbolic writing 
we get for the unidirectional case 



H{s, e) H (m) and H(t, s, e) =: H 



(6) 



In the symbolic notation of conditional entropies h(i\s) 
we will hatch t, e.g. yields 



h{t\s, e) = Hit, s, e) - His, e) 



(Eh) 



The KS-entropy of a dynamical system is defined as 
the conditional entropy of the state of the system know- 
ing the full past. Because the KS-entropy is proportional 
to the system size N in our case, we can introduce the 
entropy density rj which is the KS-entropy divided by N. 
The definition of the entropy density can be written as 
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where the entropies are calculated for a finite CML with 
N lattice sites. We rewrite the r.h.s. of in such a way 
that only a single site remains as the conditioned part in 
the conditional entropies [O: 
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(8) 



The translational invariance and periodic boundary con- 
ditions allow cyclic permutations of columns. After shift- 
ing t to the leftmost column in each single term in (|^), 
we split the blocks s into si and S2 like 
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For any S2, hit \ si S2) fulfills the inequality 
hit\s{ ) > hitlsisi) 



(9) 



It says that the uncertainty about the state t is the larger 
the less I know about the rest of the system. Formally 
this can be shown using Jensen's inequality (see e.g. |^). 
If we apply (||) to (^) we get 




< Nh I m+l| 




(10) 
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For the sake of clarity we restricted the argumentation to 
the unidirectional case. It will be obvious how the same 
reasoning can be applied to the diffusive case. 
Let us introduce the abbreviation 

hpim,l):=h(^n+i[^^ . (11) 
The index p means "pyramid" and denotes the form of 



the spatio-temporal embedding. The first argument m 
gives the number of time steps used for prediction and 
the second argument denotes the number of lattice sites 
predicted. That means e.g. 



/ip(2,3) = h 
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FIG. 1. Estimates of hp{l, 1) (stars) for the unidirectional linear coupling (left) and the diffusive nonlinear one (right) 
compared to the entropy density computed via the Pesin identity, rjx. 



Now we can formulate an upper bound for the entropy 
density: 

rj < lim —hp{m,n) . (12) 
c^o n 

Here we used again the same type of inequality like (|^) 
which leads to 

hp[m,n) < hp{m',n) if m > m' . (13) 

— hp(m,n) < — hp(m,n') if n > n' . (14) 

Similar to the usual conditional entropies in low dimen- 
sional systems the entropies hp{m,n) become constant 
on sufficiently small length scales (see e.g. Fig. ^). Thus 
the entropies hp(m,n) provide an upper bound for the 
entropy density already on finite length scales e. If one is 
interested in the quality of the bound one has to consider 
the inequalities (|l3|) , which reflect the effects of correla- 
tions in time, and (p^), which reflect the spatial corre- 
lations. Generally, the tighter the bound (^ should be, 
the less local the measurement has to be. 

While the results in the former section were derived 
for the usual Shannon entropy, for numerical investiga- 
tions the entropies based on the correlation sum are much 
more convenient. They provide better statistics and re- 
quire less computational effort. The correlation sum is 
deflned as 

Cie) = ^^^^^T.ei^~W.-s,\). (15) 



The quantity H{s,e) = — InC(e) can be regarded as 
a generalized entropy, the so called correlation entropy 
p^-p^. One disadvantage of using the correlation en- 
tropy is that inequality (^ does no longer hold rigor- 
ously. Although experience shows that the deviations 
are usually small, to our knowledge there do not exist 
theoretical arguments supporting this. A second disad- 
vantage is that the correlation entropy is a lower bound 
of the Shannon entropy. Thus, strictly speaking we can- 
not expect to estimate an upper bound of the entropy 
density by using the correlation integral. Nevertheless, 
we can interpret the results provided by the correlation 
sum as approximate estimates of the entropy density as it 
was done with the usual correlation entropies as approx- 
imate estimates for the KS-entropies of low dimensional 
systems (e.g. Q). 

Fig. |l| shows the estimates of /ip(l,l) as a function 
of the coupling a estimated by using the correlation sum 
(stars) . They are compared to the results for the entropy 
density calculated by the Pesin identity (solid line) via 
the Lyapunov exponents r]x — ^/N'^^Xi with > 0. 
The scaling with respect to e can be seen in Fig. |^ for 
the unidirectional coupling with a = 0.2 and a — 0.5. 
In the example with a = 0.2 the estimates of hp{m, 1) 
are almost independent of m and turn out to be a good 
approximation of the entropy density, calculated by the 
Pesin-identity. This coincides with the observation in 
Fig. |l| that the value of hp{l, 1) is very close to the value 
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calculated via the Lyapunov exponents for cr < 0.3 for 
the unidirectional and a < 0.05 in the diffusive case. For 
larger coupling effects of correlations become visible. In 
Fig. H one sees that hp{2, 1) is a remarkably better esti- 



mate than hp{l, 1). Further increasing m gives no better 
results. The remaining difference between hp{m, 1) and 
the Pesin value might be due to the spatial correlations 
in the system. 
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FIG. 2. Estimates of hp{m, 1) for the unidirectional case, linear coupling with a — 0.2 (left) and a = 0.5 (right). 



Moreover, in the strong coupling case the problem of 
the violation of (||) by the correlation entropies becomes 
relevant. Fig. ^ shows /ip(m, 1) for m = 1,2,3. As one 
can see the inequality is strongly violated. The rea- 
son for this behaviour is still an open question. 



provides a rather good approximation of the entropy den- 
sity 

We would like to thank Peter Grassberger for stimulat- 
ing discussions and for carefully reading the manuscript. 
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FIG. 3. hp{m, 1) m = 1, 2, 3 for the diffusive case, a — 0.3 
estimated by using the correlation entropies. 

In summary, we presented a method to estimate the 
entropy density in coupled map lattices with local cou- 
plings using only observables of local subsystems which 
corresponds to the estimates of the KS-entropy in low di- 
mensional systems. As shown in eqn. (f^)-(|To|) this would 
be an upper bound of the KS-entropy, if we could use 
an algorithm based on a partition of the phase space. 
Unfortunately, this is not possible due to the statistical 
requirements. The correlation method uses a covering, 
instead, so that the inequalities shown are no longer valid, 
rigorously. Nevertheless, the numerical investigation we 
presented for two simple cases showed that this method 
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